Usage of mobile computing devices such as smartphones, tablets, computerized wristwatches, audio players, and netbooks have increased dramatically as the capability of these devices have expanded to coincide with advances in miniaturization. Foremost among these features is the ability to execute applications and operating systems of increasing complexities. Typically, mobile computing devices are implemented with advanced integrated circuits called “system on a chip” or alternately, “system on chip” (abbreviated as “SoC”), which integrate several functions and components of a traditional computing system on a single chip. These components often include one or more central processing units (CPUs), graphics processing units (GPUs), and display controllers, which cooperatively produce the graphical output and user interfaces of the applications and operating system executing in the mobile device and displayed in the display panel(s) of the mobile computing device.
However, due to inherent limitations arising from their miniaturized size, SoCs may suffer from performance issues, particularly when processing for multiple applications becomes intensive. To alleviate this problem, dedicated hardware overlays have been developed and are often incorporated in many SoC designs. Hardware overlays are dedicated buffers into which an application can render output without incurring the significant performance cost of checking for clipping and overlapping rendering by other executing applications. An application using a hardware overlay to store output is allocated a completely separate section of video memory accessible (at least temporarily) only to that application. Because the overlay is otherwise inaccessible, the program is able to bypass verifying whether a given piece of the memory is available to the program, nor does the application need to monitor for changes to the memory addressing.
Unfortunately, display controllers inside system on a chips have limited number of overlay windows. For example, many of the current generation of SoCs have three overlay windows per display controller. However, the number of distinct graphical layers to be composited (i.e., rendered) keeps increasing as content from various sources and available functionality expands with the development of increasingly complex applications. Each graphical layer typically corresponds to an application—in some cases, multiple layers can correspond to the same application—and represents the graphical output produced by the application and displayed on the screen or display panel of the mobile computing device.
When the number of graphical layers exceeds the number of overlay windows, an overlay overflow is triggered. When this happens, two or more layers must be pre-composited (i.e., aggregated as a single layer) by a pre-compositor before the aggregated layers are sent with the remaining non-aggregated layers to the display controller. Unfortunately this pre-composition process is a very expensive operation in terms of performance, power, and memory bandwidth consumption. In particular, this layer composition path often causes large amounts of memory traffic, which increases as the number of pixels as the number of layers increases. For example, video streaming applications are extremely popular among many users of mobile computing devices. Graphical output corresponding to a video streaming application may include a region that displays streaming video, along with a separate region that contains a graphical user interface for manipulating playback of the video. Each of these regions may be implemented as a separate graphical layer. Traditionally, the video output may be decoded and produced by a video decoder, with the GUI being produced by a GPU, and stored in a frame buffer or system memory. Other applications with similar dispositions include gaming applications, which may produce graphical output contained in one or more layers.
Other common layers include status bars, navigation bars, or virtual keyboards. One common display configuration is implemented such that a status bar corresponding to the mobile computing device occupies a relatively thin portion of the display at the top or bottom of the display. Information presented in the status bar may include such information as: remaining battery life; connectivity with a data network; bluetooth operation or non-operation; time, and/or graphical icons pertaining thereto. Another display configuration typically includes a virtual keyboard occupying a portion of the bottom of the display screen. Yet another common configuration includes a navigation bar that includes statically positioned graphical icons linked to critical or frequently used applications. As these features (and their corresponding layers) are updated infrequently, the layers may be pre-composited or composited separately from more active layers (such as video streaming or gaming applications) and also stored in frame buffers and/or external memory prior to being composited in the display controller with other pending layers as a single, coherent frame of graphical content.
However, due to the spatial arrangement (e.g., the status bar may be at the top, whereas the navigation bar or virtual keyboard at the bottom of the rendered display), since the size of frame buffers correspond to the size of a display frame, an entire frame buffer may be dedicated to storing the content from the static applications, even though a substantial, or even significant majority of the display—being apportioned to display actively updating application content—contains no actual graphical content. Naturally, significant inefficiency of both memory usage, and memory access bandwidth can result from conventional layering and overlay apportionment techniques.